Unintended consequences of measures implemented in the school setting to contain the COVID-19 pandemic: a scoping review.

BACKGROUND: With the emergence of SARS-CoV-2 in late 2019, governments worldwide implemented a multitude of non-pharmaceutical interventions in order to control the spread of the virus. Most countries have implemented measures within the school setting in order to reopen schools or keep them open whilst aiming to contain the spread of SARS-CoV-2. For informed decision-making on implementation, adaptation, or suspension of such measures, it is not only crucial to evaluate their effectiveness with regard to SARS-CoV-2 transmission, but also to assess their unintended consequences. OBJECTIVES: To comprehensively identify and map the evidence on the unintended health and societal consequences of school-based measures to prevent and control the spread of SARS-CoV-2. We aimed to generate a descriptive overview of the range of unintended (beneficial or harmful) consequences reported as well as the study designs that were employed to assess these outcomes. This review was designed to complement an existing Cochrane Review on the effectiveness of these measures by synthesising evidence on the implications of the broader system-level implications of school measures beyond their effects on SARS-CoV-2 transmission. SEARCH METHODS: We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase, four non-health databases, and two COVID-19 reference collections on 26 March 2021, together with reference checking, citation searching, and Google searches. SELECTION CRITERIA: We included quantitative (including mathematical modelling), qualitative, and mixed-methods studies of any design that provided evidence on any unintended consequences of measures implemented in the school setting to contain the SARS-CoV-2 pandemic. Studies had to report on at least one unintended consequence, whether beneficial or harmful, of one or more relevant measures, as conceptualised in a logic model.  DATA COLLECTION AND ANALYSIS: We screened the titles/abstracts and subsequently full texts in duplicate, with any discrepancies between review authors resolved through discussion. One review author extracted data for all included studies, with a second review author reviewing the data extraction for accuracy. The evidence was summarised narratively and graphically across four prespecified intervention categories and six prespecified categories of unintended consequences; findings were described as deriving from quantitative, qualitative, or mixed-method studies. MAIN RESULTS: Eighteen studies met our inclusion criteria. Of these, 13 used quantitative methods (3 experimental/quasi-experimental; 5 observational; 5 modelling); four used qualitative methods; and one used mixed methods. Studies looked at effects in different population groups, mainly in children and teachers. The identified interventions were assigned to four broad categories: 14 studies assessed measures to make contacts safer; four studies looked at measures to reduce contacts; six studies assessed surveillance and response measures; and one study examined multiple measures combined. Studies addressed a wide range of unintended consequences, most of them considered harmful. Eleven studies investigated educational consequences. Seven studies reported on psychosocial outcomes. Three studies each provided information on physical health and health behaviour outcomes beyond COVID-19 and environmental consequences. Two studies reported on socio-economic consequences, and no studies reported on equity and equality consequences. AUTHORS' CONCLUSIONS: We identified a heterogeneous evidence base on unintended consequences of measures implemented in the school setting to prevent and control the spread of SARS-CoV-2, and summarised the available study data narratively and graphically. Primary research better focused on specific measures and various unintended outcomes is needed to fill knowledge gaps and give a broader picture of the diverse unintended consequences of school-based measures before a more thorough evidence synthesis is warranted. The most notable lack of evidence we found was regarding psychosocial, equity, and equality outcomes. We also found a lack of research on interventions that aim to reduce the opportunity for contacts. Additionally, study investigators should provide sufficient data on contextual factors and demographics in order to ensure analyses of such are feasible, thus assisting stakeholders in making appropriate, informed decisions for their specific circumstances.

Machine learning reduced workload for the Cochrane COVID-19 Study Register: development and evaluation of the Cochrane COVID-19 Study Classifier.

BACKGROUND: This study developed, calibrated and evaluated a machine learning (ML) classifier designed to reduce study identification workload in maintaining the Cochrane COVID-19 Study Register (CCSR), a continuously updated register of COVID-19 research studies. METHODS: A ML classifier for retrieving COVID-19 research studies (the 'Cochrane COVID-19 Study Classifier') was developed using a data set of title-abstract records 'included' in, or 'excluded' from, the CCSR up to 18th October 2020, manually labelled by information and data curation specialists or the Cochrane Crowd. The classifier was then calibrated using a second data set of similar records 'included' in, or 'excluded' from, the CCSR between October 19 and December 2, 2020, aiming for 99% recall. Finally, the calibrated classifier was evaluated using a third data set of similar records 'included' in, or 'excluded' from, the CCSR between the 4th and 19th of January 2021. RESULTS: The Cochrane COVID-19 Study Classifier was trained using 59,513 records (20,878 of which were 'included' in the CCSR). A classification threshold was set using 16,123 calibration records (6005 of which were 'included' in the CCSR) and the classifier had a precision of 0.52 in this data set at the target threshold recall >0.99. The final, calibrated COVID-19 classifier correctly retrieved 2285 (98.9%) of 2310 eligible records but missed 25 (1%), with a precision of 0.638 and a net screening workload reduction of 24.1% (1113 records correctly excluded). CONCLUSIONS: The Cochrane COVID-19 Study Classifier reduces manual screening workload for identifying COVID-19 research studies, with a very low and acceptable risk of missing eligible studies. It is now deployed in the live study identification workflow for the Cochrane COVID-19 Study Register.

Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation.

OBJECTIVE: To compare results reporting and the presence of spin in COVID-19 study preprints with their finalised journal publications. DESIGN: Cross-sectional study. SETTING: International medical literature. PARTICIPANTS: Preprints and final journal publications of 67 interventional and observational studies of COVID-19 treatment or prevention from the Cochrane COVID-19 Study Register published between 1 March 2020 and 30 October 2020. MAIN OUTCOME MEASURES: Study characteristics and discrepancies in (1) results reporting (number of outcomes, outcome descriptor, measure, metric, assessment time point, data reported, reported statistical significance of result, type of statistical analysis, subgroup analyses (if any), whether outcome was identified as primary or secondary) and (2) spin (reporting practices that distort the interpretation of results so they are viewed more favourably). RESULTS: Of 67 included studies, 23 (34%) had no discrepancies in results reporting between preprints and journal publications. Fifteen (22%) studies had at least one outcome that was included in the journal publication, but not the preprint; eight (12%) had at least one outcome that was reported in the preprint only. For outcomes that were reported in both preprints and journals, common discrepancies were differences in numerical values and statistical significance, additional statistical tests and subgroup analyses and longer follow-up times for outcome assessment in journal publications.At least one instance of spin occurred in both preprints and journals in 23/67 (34%) studies, the preprint only in 5 (7%), and the journal publications only in 2 (3%). Spin was removed between the preprint and journal publication in 5/67 (7%) studies; but added in 1/67 (1%) study. CONCLUSIONS: The COVID-19 preprints and their subsequent journal publications were largely similar in reporting of study characteristics, outcomes and spin. All COVID-19 studies published as preprints and journal publications should be critically evaluated for discrepancies and spin.

Evaluation of the comprehensiveness, accuracy and currency of the Cochrane COVID-19 Study Register for supporting rapid evidence synthesis production.

The Cochrane COVID-19 Study Register (CCSR) is a public, continually updated database of COVID-19 study references. The aim of this study-based register is to support rapid and living evidence synthesis, including an evidence ecosystem of COVID-19 research (CEOsys). In November and December 2020, we conducted an evaluation of the CCSR for CEOsys, measured its performance and identified areas for improvement. For the evaluation we generated a purposive sample of 286 studies from 20 reviews to calculate the CCSR's comprehensiveness (sensitivity), accuracy (correctly classified and linked studies) and currency (time to publish and process references). Our sample showed that the CCSR had an overall comprehensiveness of 77.2%, with the highest coverage for interventional studies (94.4%). The study register had 100% coverage for trial registry records, 86.5% for journal articles and 52.4% for preprints. A total of 98.3% of references were correctly classified with regard to study type, and 93.4% with regard to study aim. A total of 89% of studies were correctly linked. A total of 81.4% of references were published to the register in under 30 days, with 0.5 day (median) for trial registry records, 2 days for journal articles and 56 days for preprints. The CCSR had high comprehensiveness, accurate study classifications and short publishing times for journal articles and trial registry records in the sample. We identified that coverage and publishing time for preprints needed improvement. Finally, the evaluation illustrated the value of a study-based register for identifying additional study references for analysis in evidence synthesis.